Developer Guide

This document explains how to reuse the chat and document-grounded QA patterns from this example in other desktop apps.

Install and Set Up LM Studio

1. Install LM Studio from the official website. (https://lmstudio.ai/)
2. Download and load at least one local chat model. You sould do this in LM Studio. (Tested with qwen3-4b-instruct-2507)
3. Start the local LM Studio server.
4. Verify endpoints:
   • GET http://localhost:1234/v1/models
   • POST http://localhost:1234/v1/chat/completions
5. Configure your app to use BaseURL = http://127.0.0.1:1234 (or your custom host/port).

For document extraction in this example, install pdftotext (PDF) and pandoc (DOCX) if you need those formats.

Core Pieces to Reuse

1. API client class: LMStudioAPI (HTTP wrapper for local model calls).
2. Chat orchestration: send user messages, receive completions, append to transcript.
3. Continuation loop: auto-request continue chunks for long outputs.
4. RAG layer: attach files, extract text, chunk/index, retrieve, augment prompt.

LMStudioAPI Integration Contract

Instantiate one API client and set at minimum:

• BaseURL (default: http://127.0.0.1:1234)
• Model (set from available models)
• MaxTokens, PerRequestMaxTokens, Temperature, TopP
• Memory = True for continuation-based long outputs

Handle events:

• ResponseReceived("available.models", data)
• ResponseReceived("chat.completion", messages)
• ErrorReceived(url, status, content)

Chat Flow

onUserSend(message):
  setWorking(true)
  add user line to transcript
  outbound = buildPrompt(message)   // plain chat or RAG-augmented
  api.AddMessage("user", outbound)
  api.SendChatCompletionRequest()

onChatCompletion(reply, finishReason):
  append assistant chunk
  if shouldContinue(finishReason, chunkSize):
    api.AddMessage("user", "Continue exactly where you left off...")
    api.SendChatCompletionRequest()
  else:
    close assistant block
    setWorking(false)

Keep a hard max continuation count to avoid runaway loops.

RAG Pipeline

1. Collect user-selected files.
2. Enforce limits (example: max 5 files, max 30MB total).
3. Extract text by extension.
4. Chunk text with overlap (example: 1200 chars, 200 overlap).
5. Store chunks[] and matching sources[].
6. At query time, score chunks against question terms.
7. Inject top chunks into prompt.

outbound =
  "Use retrieved excerpts to answer. If missing, say not found." +
  "\n\nQuestion:\n" + question +
  "\n\nRetrieved Excerpts:\n" + topChunksText

Recommended retrieval safety checks:

• Always bounds-check parallel arrays (chunks / sources).
• Skip empty chunks.
• Return gracefully when no matches are found.

File Extraction Strategy

• .txt / .csv: direct file read.
• .pdf: external pdftotext.
• .docx: external pandoc.

If external tools are unavailable, skip file with a clear warning and keep app responsive.

Porting Checklist

1. Copy the API client class and expose model settings in your UI.
2. Wire completion/error events to your transcript view.
3. Add a setWorking(true/false) path around request lifecycle.
4. Add continuation guardrails (max continuations + final stop conditions).
5. Implement attach/index/retrieve functions and prompt augmentation.
6. Ensure transcript always auto-scrolls on append.

Tuning Knobs

• PerRequestMaxTokens: lower for faster chunk turnaround.
• kAutoContinueMax: higher for longer aggregate outputs.
• kTopChunks: increase for broader context, decrease for tighter grounding.
• kChunkChars / kChunkOverlapChars: tune retrieval granularity.
• kLikelyTruncatedChars: fallback continuation heuristic threshold.

Failure Handling Recommendations

• Never crash on missing keys in JSON responses.
• Catch and report extraction/indexing errors per file.
• Turn off busy indicator in every terminal path (success, capped continuation, error).
• Log server URL/status/body on request failures for diagnostics.